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(57) ABSTRACT 

An AV device controller for controlling networked devices 
with a graphical user interface (GUI). An interface section 
obtains GUI information from a controlled device. A GUI 
storage section stores the GUI information. A GUI display 
section displays GUI parts. An input device selects the GUI 
parts. By selecting a GUI part displayed on the GUI display 
section with the input device, a predetermined processing 
assigned to the GUI part is started. 

10 Claims, 9 Drawing Sheets 
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AV DEVICE CONTROL APPARATUS 

BACKGROUND OF THE INVENTION 

1. Field of the Invention 

The present invention relates to a controller for various 
kinds of networked AV devices and a method of displaying 
a graphical user interface on the controller. 

2. Related Art of the Invention 

Conventional AV device controllers are included in each 
individual television or video cassette recorder (VCR), and 
the number of buttons on the controllers has increased with 
improvement in the performance of the AV devices. A 
method to control a plurality of AV devices with one 
controller is to control a television or a VCR by obtaining an 
ID code from the television or the VCR and identifying the 
ID as described in Japanese Patent Apphcation No. Hei 
9-68095. 

Under these circumstances, the following problems arise: 
Since a plurality of AV devices are present in each home 
and the controllers included therein are also present, it is 
dif&cult to identify the AV device to which each controller 
belongs. In addition, with improvement in the performance 
of the AV devices, the controllers have become compHcated 
and difficult to use. 

Even when the controllers are integrated into one con- 
troller as described in Japanese Patent Apphcation No. Hei 
9-68095, it is difficult to identify the function assigned to 
each button. 

To solve these problems, an object of the present inven- 
tion is to provide a controller capable of controlling a 
plurality of AV devices with one controller, and a method of 
displaying a graphical user interface on the controller, said 
method being characterized in that anyone can easily operate 
AV devices or the user can customize the graphical user 
interface. 

SUMMARY OF THE INVENTION 

According to a first aspect of the present invention, to 
solve the above-mentioned problems, means is provided for 
obtaining, as GUI information of controlled devices on an 
AV device network, GUI image information, GUI display 
position information and information on commands trans- 
mitted to the controlled devices. According to the first aspect 
of the present invention, since the GUI information of 
various controlled devices on the AV device network can be 
obtained, the devices on the AV device network can be easily 
controlled with one controller. 

According to a fourth aspect of the present invention, to 
solve the above-mentioned problems, the following methods 
are provided: a method in which levels are provided for GUI 
display and the user can make level selection; and a method 
in which parts are displayed in accordance with the selected 
level. According to the fourth aspect of the present 
invention, since the user selects from among, for example, 
levels from a level to display minimum necessary GUIs such 
as playback, stop and rewind of a VCR to a level to display 
a multiphcity of GUIs such that all the functions of a VCR 
are displayed and inputs the selected level and after the 
selected level is input, the desired GUIs are displayed at the 
display level, a controller being easy for the user to use can 
be constructed. 

According to a fifth aspect of the present invention, to 
solve the above-mentioned problems, the following methods 
are provided: a method in which levels are provided for GUI 
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display and the user can continuously and flexibly make 
level selection; and a method in which display is performed 
according to the selected level. According to the fifth aspect 
of the present invention, since operations of zoom in and 

5 zoom out, for example, between a level to display minimum 
necessary GUIs such as playback, stop and rewind of a VCR 
and a level to display a multiplicity of GUIs such that all the 
functions of the VCR are displayed can be freely performed 
and GUIs are displayed in GUI positions desired by the user 

10 at a display level desired by the user, a controller being easy 
for the user to use can be constmcted. 

According to a sixth aspect of the present invention, to 
solve the above-mentioned problems, a method in which the 
user can freely customize the layout of the displayed GUIs 
is provided. According to the sixth aspect of the present 
invention, since the user can move GUI parts to which 
functions such as playback, stop and rewind of a VCR are 
assigned to desired positions on the GUI display screen, a 
controller being easy for the user to use can be constructed. 

20 

According to a seventh aspect of the present invention, to 
solve the above-mentioned problems, means is provided for 
obtaining, as GUI information of a plurality of controlled 
devices on an AV device netwo±, GUI image information, 

25 GUI display position information, information on com- 
mands transmitted to the controlled devices and the device 
identifiers of the controlled devices, and a method is pro- 
vided in which the user can fireely customize the layout of 
GUI parts of a plurality of devices. According to the seventh 

30 aspect of the present invention, for example, on an AV 
network to which a CD player (CD) and a minidisk player 
(MD) are connected, the user can freely dispose the GUI part 
to which the recording function of the MD is assigned next 
to the GUI part to which the playback function of the CD is 

35 assigned. As described above, an AV network controller can 
be constructed that is easy for the user to use and in which 
all the devices on an AV device network are controlled with 
one controller. 

40 BRIEF DESCRIPTION OF THE DRAWINGS 
[FIG. 1] 

A block diagram of a controller according to a first 
embodiment of the present invention; 

[FIG. 2] 

A block diagram of a controller according to a second 
embodiment of the present invention; 

[HG. 3] 

50 

A view of the screen of a graphical user interface accord- 
ing to the second embodiment of the present invention; 

[FIG. 4] 

A block diagram of a controller according to a third 
embodiment of the present invention; 

[FIG. 5] 

A view of the screen of a graphical user interface accord- 
ing to the third embodiment of the present invention; 

60 

[FIG. 6] 

A block diagram of a controller according to a fourth 
embodiment of the present invention; 

65 [PIG. 7] 

A view of the screen of a graphical user interface accord- 
ing to the fourth embodiment of the present invention; 
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[FIG. 8] 

A view showing an example of the cxjnfiguration of GUI 
information in the first embodiment of the present invention; 
and 

[HG. 9] 

A view showing an example of the configuration of GUI 
information in a fifth embodiment of the present invention. 

DESCRIPTION OF SYMBOLS 

101 Controller 

102 Interface section 

103 GUI storage section 

104 GUI display section 

105 Input means 

201 GUI level setting section 

202 GUI position setting section 

301 Level setting screen 

302 Zoomed out GUI display screen 

303 Zoomed in GUI display screen 

304 GUI level selection bar 

305 Zoom-out selection 

306 Zoom-in selection 
401 Level selecting means 

501 2^omed out GUI display screen 

502 Zoomed in GUI display screen 

503 Position selection 

504 Continuous progress 
601 GUI part position storage section 

701 GUI display screen before a GUI part is moved 

702 GUI di^lay screen after a GUI part is moved 

703 GUI part before moved 

704 GUI part after moved 

801 File name of GUI part 

802 Display position of GUI part 

803 Display size of GUI part 

804 Processing content 

805 Identifier of GUI part 

806 GUI part storage table 

901 File name of GUI part 

902 Display position of GUI part 

903 Display size of GUI part 

904 Processing content 

905 Identifier of GUI part 

906 Identifier of device 

907 GUI part storage table 

PREFERRED EMBODIMENTS OF THE 
INVENTION 

(First Embodiment) 

Hereinafter, a controller according to a first embodiment 
of the present invention will be described with reference to 
the drawings. 

FIG. 1 is a block diagram of the controller according to 
the first embodiment of the present invention. In FIG. 1, 
reference numeral 101 represents the controller, reference 
numeral 102 represents an interface section, reference 
numeral 103 represents a GUI storage section, reference 
numeral 104 represents a GUI display section, and reference 
numeral 105 represents input means. Reference numeral 
1000 represents various kinds of AV devices connected 
through the interface section 102. 

FIG. 8 shows an example of the configuration of the GUI 
information stored in the GUI storage section 103. In FIG. 
8, reference numeral 801 represents the file name of the 
image data of a GUI part, reference numeral 802 represents 
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the display position of the GUI part, reference numeral 803 
represents the display size of the GUI part, reference 
numeral 804 represents the processing content, reference 
numeral 805 represents the identifier of the GUI part, and 
reference numeral 806 represents a GUI part storage table. 

The operation of the controller of the embodiment struc- 
tured as described above will be described. 

The controller 101 first obtains the GUI information by 
way of the interface section 102 firom the ROM of the 
controlled AV device on the network. When the controlled 
device is a VCR, the GUI information includes the image 
data of GUI parts such as playback, stop and rewind, the file 
names 801 thereof, the display position data 802 of the 
image data, the display size data 803 of the image data, and 
the description of the processing 804 performed when the 
displayed GUI parts are selected. 

The controller 101 stores the GUI information into the 
GUI part storage table 806 in the GUI storage section 103, 
When the GUI information is stored into the GUI part 
storage table 806, the GUI part identifiers 805 are uniquely 
assigned. 

The GUI display section 104 displays GUI parts with 
reference to the file names 801, the display position data 802 
and the display size data 803 of the GUI parts in the GUI 
storage section 103. The GUI display section 104 stores the 
identifiers 805 of the displayed GUI parts. 

The user can select a GUI part displayed on the GUI 
display section 104 by moving a movement button on the 
input means 105 within the selectable GUI parts and using 
an enter button on the input means 105. 

Based on the identifier 805 of the selected GUI part, the 
GUI display section 104 searches for the corresponding GUI 
part in the GUI part storage table 806, and decides the 
processing with reference to the accompanying processing 
content. When the processing is decided, a command such as 
playback, stop or rewind is transmitted by way of the 
interface section 102 to the controlled device, so that the 
controlled device starts the processing. 

(Second Embodiment) 

Hereinafter, a graphical user interface method according 
to a second embodinent of the present invention will be 
described with reference to the drawings. 

FIG. 3 shows an example of the graphical user interface 
method according to the second embodiment of the present 
invention. In FIG. 3, reference numeral 301 represents a 
level setting screen, reference numeral 302 represents a most 
zoomed out GUI display screen, and reference numeral 303 
represents a most zoomed in GUI display screen. 

On the level setting screen 301, the user changes the 
display level of the GUI for controlling AV devices by 
selecting a level from a GUI level selection bar 304. In order 
that as many GUI parts as possible are displayed on the 
screen, a lowest level 305 is selected to change the display 
to the display 302 of the whole of the GUI. In order that as 
few GUI parts as possible are displayed being enlarged, a 
highest level 306 is selected to change the display to the 
local display 303 of the GUI. 

In this example, when the display is enlarged, some of the 
parts are enlarged, and with respect to the selection, that is, 
the display positions of the enlarged parts, in the case of a 
VTR, the zoom-in and zoom-out positions are allocated to 
the level selection bar 304 so that the playback and the stop 
parts which are frequently used control GUI parts are in the 
center. Thus, the user's convenience is sufficiently consid- 
ered. 
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FIG. 2 is a block diagram of a controller when the stored in the GUI storage section 103, the selected GUI 

graphical user interface method according to the second display level and the GUI display position, 

embodiment of the present invention is carried out. In FIG. (Fourth Embodiment) 

2, in addition to the elements of FIG. 1, reference numerals tt • ^ u- i •* ^ ^^iu^A 
201 in the GUI display section 104 represent a GUI level 5 , HeremaAer, a graphical user mterface 

setting section, and reference numer^s 202 in the GUI ^ ^ f^^^ embodiment of the present mvenUon wiU be 

display section 104 represent a GUI position setting. The descnbed with reference to the drawmgs. 

operation of the controller structured as described above will FIG. 7 shows an example of the graphical user interface 

be described. method accordiag to the fourth embodiment of the present 

The GUI displays GUI parts on the GUI display section invention. In FIG. 7, reference numeral 701 represents a 

104 based on the GUI information obtained in the first GUI display screen before a GUI part is moved, reference 

embodiment. The GUI display section 104 is capable of numeral 702 represents a GUI display screen after a GUI 

displaying the level setting screen in zoom in and zoom out part is moved, reference numeral 703 represents a GUI part 

of the GUI display. On the level setting screen, the levels, the before moved, and reference numeral 704 represents a GUI 

parts to be displayed when the levels are selected and the part after moved. 

display coordinates thereof are suppHed to the GUI position ^ ^.^I display screen 701, the user can change the 

settmg section 202. , , , . , position of the GUI part 704 by selecting and moving the 

The GUI level, the parts displayed m correspondence ^.^I part 703. The position is stored in a GUI part position 

therewith and the positions thereof can be changed by the ^^^^^^^ ^^^^^^ ^ j ^^^^^ ^^^^ 

user. Specifically, on the level settmg screen 301, the user , „^c^t,v^« nV^arMr^ 
selects an arbitrary GUI level from the GUI level selection 20 position cnange. 

bar 304 with the enter button existing on the input means FIG. 6 is a block diagram of a controller when the 
105, and then, on the GUI display screen, the user moves a graphical user interface method according to the fourth 
displayed GUI part with the movement button existing on embodiment of the present invention is carried out. In FIG. 
the input means 105, and decides the display position by 6» in addition to the elements of the controUer of FIG. 4, 
depressing the enter button. The result is recorded into the ^ reference numeral 601 in the GUI display section 104 
GUI level setting section 201 and the GUI position setting represents the GUI part position storage section. The opera- 
section 202. controller structured as described above will be 
(Third Embodiment) described. 

Hereinafter, a graphical user interface method according GUI parts are displayed on the GUI display section 104 

to a third embodiment of the present invention will be based on the GUI information obtained m the first embodi- 

described with reference to the drawings. ^ent. The cursor 503 is displayed on the GUI display section 

FIG. 5 shows an example of the graphical user interface and the GUI display section 104 obtains the position 

method according to the third embodiment of the present information of the cursor 503. The user moves the oirsor 
invention. In FIG. 5, reference numeral 501 represent a „ 503 with the movement button on the mput means 105, and 

most zoomed out GUI display screen, and reference numeral depresses the enter button on the mput means 105 when the 

502 represents a most zoomed in GUI display screen. <^^' ^03 is on a specific GUI part. By domg this, the user 

On the zoomed out GUI display 501, the user can con- ^^^^^^ ^P^^^ P^' 

Unuously progress 504 the zoom in or zoom out of the GUI Under this state, the user moves the position of the GUI 

display by freely selecting the position by moving a cursor part with the input means 501 and depresses the enter button 

503, and continuously increasing or decreasing the GUI once again, so that the GUI part is moved to the new 

level at the same time. In the example of FIG. 5, the user position. The new position of the GUI part is stored into the 

indicates a desired position of the GUI display 501 with the GUI part position storage section 601. That is, the relative 

cursor 503 and at the same time, zooms in the GUI display positional interrelationship among the GUI parts is newly 

to the GUI display 502 with level selecting means 401, stored. 

FIG. 4 is a block diagram of a controUer when the The GUI display section 104 changes the displayed GUI 

graphical user interface method according to the tiiird based on the GUI part position recorded in the GUI part 

embodiment of the present invention is carried out. In FIG. position storage section 601 in addition to the GUI infor- 

4, in addition to the elements of the controller of FIG. 2, mation stored in the GUI storage section 103, the GUI 
reference numeral 401 represents the level selecting means. 50 display level stored in the GUI level setting section 201 and 

The operation of the controller structured as described above the GUI display position recorded in the GUI position 

will be described. setting section 202 as desaibed above. 

The GUI displays GUI parts on the GUI display section (Fifth Embodiment) 

104 based on the GUI information obtained in the first Hereinafter, a graphical user interface method according 
embodiment. The cursor 503 is displayed on the GUI display 55 to a fifth embodiment of the present invention will be 

section 104, and the GUI display section 104 can obtain the described with reference to the drawings, 

position information of the cursor 503. piG. 9 shows an example of the configuration of the GUI 

The user moves the cursor 503 with the movement button information stored in the GUI storage section 103. In FIG. 

on the input means 105 and specifies the zoom-in or zoom- 9, reference numeral 901 represents the file name of the 
out position, and the coordioate value of the position is 60 image data of a GUI part, reference numeral 902 represents 

recorded into the GUI position setting section 202. At the the display position of the GUI part, reference numeral 903 

same time, the user sets the degree of zoom in or zoom out represents the display size of the GUI part, reference 

with the level selecting means 401, and the numerical value numeral 904 represents the processing content, reference 

of the GUI display level is recorded into the GUI level numeral 905 represents the identifier of the GUI part, and 
setting section 201. 65 reference numeral 906 represents the identifier of a device, 

Ihe GUI display section 104 is capable of continuously and reference numeral 907 represents a GUI part storage 

changing the displayed GUI based on the GUI information table. 
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The GUI information is obtained by way of the interface of the functions of all or some of the above-described means 

section from the ROM of a plurality of controlled AV of the present invention, and is a computer-readable medium 

devices on the network. The GUI information includes the in ^ich the read out program and/or data performs the 

image data of GUI parts and the file names 901 thereof, the functions in cooperation with the computer, 
display position data 902 of the image data, the display size 5 The data includes the data configuration, the data format, 

data 903 of the image data, the processing 904 performed the data kind, and so on. 

when the displayed GUI parts are selected, and the device medii^ includes recording media sudi as ROMs, 

identifier 906 for identifying a plurality of AV devices. The transmission media such as the Internet and transmission 

GUI mformation is stored into the GUI part storage table j^^^ia such as light, radio waves and sound waves. 

907 in the GUI storage section 103. lo . , A^^A^^ ^ a- 

. * . The holding medium includes, for example, recording 

When the GUI information is stored into the GUI part media where a program and/or data is recorded, transmission 

storage table 907, the GUI part identifiers 905 are uniquely ^^^^ transmitting a program and/or data, and the like, 
assigned. The GUI display section 104 displays GUI parts ti . , , . ♦ • j- ♦ r i 

r ^ ^TTT ^ . . Li Bemg processable by a computer indicates, for example, 

with reference to the GUI part storage table 907. i. • j ui l * • *i- r j- j- 

^ ^ beingreadableby a computer m the case of recording media 

In order to prevent the GUI parts of a plurality of AV ^^^^ ^^^^ includes that the program and/or data to 
devices firom overlapping one another in the mitial display transmitted can be handled by a computer as a result of 

when the GUI information is obtained, an area in the display transmission in the case of transmission media, 
screenisobtainedfor each AV device and the GUI parts are a-^ *-iJi? i a. 

j i 1- ^i-juAT/j- An information aggregate includes, for example, software 

displayed. That is, m a case where two kmds of AV devices, ^^^^ ^ proeram md/or data 

a VCR and a CD are present, as the display position 902 of ^ ,* . . . 

the GUI part of the VCR, the value obtained by dividing the . ^ f "^l*"^^ f ^ccordrng to the present mvenUon, 

display position data obtained from the ROM by 2 is stored ^"^^ ^! ^UI information of the AV devices on the netwoik 
in the GUI part storage table 907, and as the display size 903, ''^ obtamed, the AV devices can be easily controlled with 

the value reduced to a quarter is stored. On the other hand, controller. 

as the display position 902 of the GUI part of the CD, the ^5 Moreover, according to the present invention, since the 

sum of the value obtained by dividing the display position display level of zoom in and zoom out of the GUI can be 

data obtained from the ROM by 2 and the vahie shifted by changed, it is possible to serve all the needs from the need 
half the screen in a lateral direction of the screen is stored in user requiring GUI parts for minimum necessary control 

the GUI part storage table 907, and as the display size 903, to the need of a user requiring a multiplicity of GUI parts for 

the value reduced to a quarter is stored. efSciendy using a multiplicity of functions. 

When a device is newly added to the network, the Moreover, according to the present invention, since the 

displayed GUI parts can also be prevented from overlapping display level of zoom in and zoom out of the GUI can be 

one another by re-calculating the data of the display position continuously changed, in devices with a limited display 

902 and the data of the display size 903 already stored in the space such as portable terminals, the GUI display can be 

GUI part storage table 907. nsed without any difference from when it is provided on a 

In a case where the display screen is divided into a ^^S^ screen. Moreover, it is possible to serve all the needs 
plurality of areas and the GUI parts of a plurality of AV the need of a user requiring GUI display for minimum 
devices are displayed in each area respectively as described necessary control to the need of a user requiring a multi- 
above, when the sizes of the GUI parts are not proportionally P^^^^y ^^1 displays for effidenay using a multipHcity of 
reduced in area but, for example, the screen is divided into functions. 

4/1 areas and the GUI parts are displayed in one of the areas. Moreover, according to the present invention, since the 

it is desirable to display the GUI parts so as to be largish iiser can freely customize the layout of the GUI, by com- 

within the limit that is not larger than the area, such that the bining the zoom-in function and the zoom-out function at 

size is preset not to but to VS. the same time, an AV device controller serving users having 

The areas into which the screen is divided are not nec- Purposes can be constructed, 

essarily square or rectangular, but may be circular or of a Moreover, according to the present invention, since the 

bent shape, or the like. GUI information of a plurality of AV devices on the network 

It is also desirable that the colors or the like of the parts can be obtained, the GUIs of a plurality of AV devices can 

be different among the devices so that a plurality of AV be simultaneously displayed with one controller, 
devices can be distinguished. What is claimed is: 

When a GUI part is selected, the corresponding GUI part ^ ^ ^ ^^^^^^ controller for controlUng networked 

in the GUI part storage table 907 is searched for based on the devices with a graphical user interface (GUQ, comprising: 
selected GUI part identifier 905, and the processing is an interface section for obtaining GUI information from a 
decided with reference to the accompanying processing 55 controlled device; 

content. When the process is decided, a command is trans- a GUI storage section for storing the GUI information; 
mitted by way of the interface section to the device a GUI display section for displaying the GUI parts; 

described in the device identifier 906, and the device to input means for selecting the GUI parts; and 
which the command is transmitted starts the processing. a GUI part position storage section capable of freely 

In the graphical user interface method of the fifth eo changing a relative positional relationship among the 
embodiment, since the method of the fourth embodiment GUI parts displayed on said display section and of 

can be applied to the GUI display of a plurality of AV storing the relationship, wherein 

devices, when the GUI parts of a plurality of devices are a user can customize the GUI, 

intermixed in the GUI display section, the GUI parts can be the GUI parts are GUI controlHng parts for controlling 

moved to respective positions desired by the user. 65 the networked devices, and 

The present invention is a medium holding a program by selecting a GUI controlling part displayed on said 

and/or data for allowing a computer to perform all or some GUI display section with said input means, prede- 
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termined processing assigned to the GUI controlling 
part is started. 

2. An AV device controller according to claim 1, wherein 
said GUI parts are displayed so as not to overlap each other 
on a screen of said GUI display section for each device. 5 

3. An AV device controller according to claim 2, wherein 
said screen is divided into areas and the GUI parts of the 
devices are displayed in the areas so that the GUI parts of 
one device are displayed in one area. 

4. An AV device controller according to claim 3, wherein lo 
the GUI parts of one device can be distinguished from the 
GUI parts of another device by their colors by a user. 

5. An AV device controller according to claim 1, wherein 
when a predetermined GUI part is specified with said input 
means, the processing corresponding to the GUI part is 15 
performed. 

6. An AV device controller according to claim 1, wherein 
said GUI storage means is capable of storing the GUI 
information so that a plurality of AV devices can be 
distinguished, and the GUI parts of a plurality of AV devices 20 
controlled by said GUI display section can be displayed so 

as to be intermixed. 

7. A medium holding a program and/or data for allowing 
a computer to perform all or some functions of all or some 

of said means of the present invention according to claim 6 25 
said medium being processable by a computer. 

8. An information aggregate being a program and/or data 
for allowing a computer to perform all or some functions of 
all or some of said means of the present invention according 
to claim 6. 
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9. An AV device controller for controlling networked 
devices with a graphical user interface (GUI), comprising: 

an interface section for obtaining GUI information from a 
controlled device; 

a GUI storage section for storing the GUI information; 

a GUI display section for displaying the GUI parts; and 

input means for selecting the GUI parts, 

a GUI part position storage section capable of freely 
changing a relative positional relationship among the 
GUI parts displayed on said display section and of 
storing the relationship, wherein a user can customize 
the GUI, 

wherein by selecting a GUI part displayed on said GUI 
display section with said input means, predetermined 
processing assigned to the GUI part is started, 

wherein said GUI display section comprises a GUI level 
setting section for performing zoom-in setting to pro- 
vide screen display so that the GUI parts are partly 
enlarged and zoom-out setting to display GUI parts in 
the whole of the screen, and a GUI position setting 
section for deciding display positions of the GUI parts 
of the zoom-in or zoom-out setting corresponding to a 
display level of the GUI parts. 

10. An AV device controller according to claim 9, com- 
prising level selecting means capable of continuously select- 
ing the display level of the GUI parts, and selecting means 
for selecting display positions of the GUI parts. 

« * * * * 
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(57) / ABSTRACT 

An apparat^ for use within a video conferencing system 
comprising an image analysis circuit and a control circuit is 
presente^TAs described herein, the image analysis circuit is 
operatwe to analyze at least a video fi-ame of video confer- 
encing signals of a video conference. The control circuit, 
coumed to the image analysis circuit, is operative to control 
a lemote video camera of a remote conferencing apparatus 
ommunicatively coupled to the video conferencing appa- 
ratus to conduct the video conference, responsive to the 
results of said analysis. 
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